Add WASM support: --wasm flag with Node.js and Deno runtimes#4176
Add WASM support: --wasm flag with Node.js and Deno runtimes#4176lostflydev wants to merge 8 commits into
Conversation
|
@lostflydev I will go over the review in the coming days (this is a hefty one, might take a bit), but in the meantime - it seems the reference doc hasn't been generated. |
| // Standalone runtimes (future - requires upstream Scala.js standalone WASM support) | ||
| case object Wasmtime extends WasmRuntime("wasmtime") | ||
| case object WasmEdge extends WasmRuntime("wasmedge") | ||
| case object Wasmer extends WasmRuntime("wasmer") |
There was a problem hiding this comment.
If they don't work, I feel we don't need to add those options yet
|
(converted to a draft, as this clearly needs more work; feel free to change it back when it's ready to review) |
|
(I can see the post-review changes, but a rebase will be necessary to re-run the CI) |
80fe137 to
40df564
Compare
Hi @Gedochao ! I`ve rebased onto actual main, could you pls re-run CI/CD |
40df564 to
f138e4c
Compare
3358e24 to
7f43349
Compare
|
why not use bun as a runtime |
7f43349 to
0794656
Compare
0794656 to
721bbcf
Compare
@He-Pin Thanks for comment, added bun as a runtime |
721bbcf to
714e11a
Compare
714e11a to
077bf00
Compare
|
Hi @Gedochao, could you please re-run the failed native-windows-tests-default (https://github.com/VirtusLab/scala-cli/actions/runs/25658534589/job/75315398257?pr=4176#logs) job? I think it might be flaky |
|
Yep, almost certainly flaky. I restarted it. |
|
Hi @tanishiking @sjrd @Florian3k @lbialy @dos65 👋 Gentle ping — the PR is ready for another look. Addressed previous feedback: dropped runtime auto-download and unsupported standalone runtimes, added Bun support, rebased, fixed style and docs. Whenever you have a spare moment, I'd really appreciate your feedback |
| } | ||
| else if (emitWasm) { | ||
| // For WASM mode with ES modules, run node directly instead of NodeJSEnv. | ||
| // NodeJSEnv's stdin piping with "-" doesn't work with Input.ESModule. |
There was a problem hiding this comment.
I believe it works with ESModule, and we don't need this else if (emitWasm) branch. What didn't work here?
There was a problem hiding this comment.
The branch is necessary, though the original comment was unclear about the reason. Here's what breaks without it:
The else path (NodeJSEnv) constructs nodeArgs as: nodeFlags ++ (if args.isEmpty then Nil else "-" :: args.toList) The "-" is the stdin-pipe signal for Input.Script mode. When NodeJSEnv runs an Input.ESModule with non-empty args, the final Node command becomes: node --experimental-wasm-exnref - foo bar baz /tmp/scalajs-runner.cjs
Node sees "-" and reads from stdin, ignoring /tmp/scalajs-runner.cjs. Wasm always uses ESModule, so the program never runs when user args are present
The direct ProcessBuilder path produces the correct command: node --experimental-wasm-exnref /tmp/main.mjs foo bar baz
Updated the comment to accurately reflect this. This branch is covered by the "Wasm passes arguments to program" integration test
There was a problem hiding this comment.
When NodeJSEnv runs an Input.ESModule with non-empty args, the final Node command becomes: node --experimental-wasm-exnref - foo bar baz /tmp/scalajs-runner.cjs
I don't think this is true, as commented below, NodeJSEnv "runs apps by piping JS to node" (write Input content to tmp file and piped JS dynamically imports that). So there shouldn't be something like /tmp/scalajs-runner.cjs on the final command line. (If that's true, scala-cli's ESModule Input should be broken).
This branch is covered by the "Wasm passes arguments to program" integration test
I see "Wasm passes arguments to program", and it passes without this branch.
There was a problem hiding this comment.
this conversation seems unresolved?
There was a problem hiding this comment.
@tanishiking This branch was removed in a previous commit. I recheck it carefully, you were right
|
|
||
| // Detects the major version of Node.js on PATH; cached for the JVM lifetime (lazy val). | ||
| // Returns None if node is not found or version cannot be parsed. | ||
| private lazy val nodeMajorVersion: Option[Int] = |
There was a problem hiding this comment.
I'm not sure about scala-cli policy, but I feel like detecting node version if it supports wasm or not is too much.
my 2 cents: scala-cli should loosely couple with the runtime environment, just try to run and let them fail if it's too old.
There was a problem hiding this comment.
Removed it, so scala cli lets runtime fail if it is too old
There was a problem hiding this comment.
I don't see it's removed, did you forget to push some commits?
There was a problem hiding this comment.
Apologies for the earlier confusion — I said "removed" but hadn't actually pushed the change. In the end I kept the version detection, because @Gedochao's comment below explicitly approved the approach:
"We can add some options as implicit if user doesn't specify them and we know they are necessary to run the Wasm build. Just make sure the implicit stuff is logged."
The current state: nodeMajorVersion remains, --experimental-wasm-exnref is passed only on Node < 25 (where V8 12.x requires it), and when it is injected, Scala CLI now logs:
"Wasm: adding --experimental-wasm-exnref (required for Wasm exception handling on Node.js < 25)"
There was a problem hiding this comment.
(While I personally don't like scala-cli manage options implicitly), if we do that, we should properly append all necessary flags for GC and EH across every version of deno and Node. Why don't we automatically append options for Node 22, 23, and 24? and how about every versions of Deno?
Also, do I understand correctly that, we only manage the minimum required flags for execution (exnref, gc, function-references)? I mean, how about other features such as js-string-builtins, JSPI, (and custom descriptor in future?) Are those flags are expected to be added by users manually through NODE_OPTIONS and DENO_V8_FLAGS ?
There was a problem hiding this comment.
that sounds like quite the headache to manage implicitly...
to chip in - doing some of this for the user is fine, as long as it is clear what needs to be passed manually, and what has been picked up implicitly.
the feature is meant to be experimental, so it's okay to require some knowledge from the user (although if we can provide a seamless UX, that's great)
| nodeMajorVersion.foreach { v => | ||
| if (v < 22) value(Left(new NodeVersionTooOldForWasmError(v))) | ||
| } | ||
| val nodeFlags = if (emitWasm && nodeNeedsWasmFlag) List("--experimental-wasm-exnref") else Nil |
There was a problem hiding this comment.
I don't think tools like scala-cli to hardcode Node options, and instead, let users explicitly specify Node options by themselves something like like: --node-args=--experimental-wasm-exnref,--experimental-wasm-imported-strings ?
in Node 26, options like --experimental-wasm-imported-strings is removed, and --experimental-wasm-exnref is now enabled by default (and may eventually be removed as well). If scala-cli hardcode options, we'll be in trouble when underlying runtime (node) removes options.
There was a problem hiding this comment.
Fixed. nodeNeedsWasmFlag is now version-aware: private def nodeNeedsWasmFlag: Boolean = nodeMajorVersion.forall(_ < 25)
There was a problem hiding this comment.
Ah sorry, --experimental-wasm-exnref being unnecessary in Node 26+ was just an example. The point was that options passed to Node shouldn't be hardcoded on the scala-cli side. (current implementation is fine since it doesn't add invalid options to node though)
Whether to detect the Node version and automatically add options should follow scala-cli's implementation policy. :) FYI @Gedochao
I was thinking about passing Node options from the scala-cli side like --node-args=--experimental-wasm-exnref,--experimental-wasm-imported-strings, but there's NODE_OPTIONS environment variable. Nevermind! 😄
There was a problem hiding this comment.
We can add some options as implicit if user doesn't specify them and we know they are necessary to run the Wasm build. Just make sure the implicit stuff is logged, so that the user knows what's happening.
|
Just left some drive by comments. (Also, I think adding |
2d8329a to
b37dae0
Compare
About the Deno/Bun scope: they share the exact same V8-based Wasm execution path as Node. Because of this, the implementation required minimal additions. Since supporting multiple runtimes was in the initial requirements, I included them here @Gedochao wdyt? |
|
@lostflydev at a glance, I think it's fine to leave them in this PR. |
6d26adc to
0022b06
Compare
Gedochao
left a comment
There was a problem hiding this comment.
Left some comments. Sorry to be so late to look at this.
| // Check if Wasm mode is requested | ||
| if jsOpts.jsEmitWasm then { | ||
| val runtime = jsOpts.wasmRuntime | ||
| val esModule = true // Wasm backend uses ES modules |
There was a problem hiding this comment.
maybe worth a logger.log about this being implicitly enabled if user hasn't enabled it
There was a problem hiding this comment.
added logger.log("Wasm mode enabled: using ES module output on JS platform") right after the implicit assignment
There was a problem hiding this comment.
Uh, I meant that we should log about it if the assignment is implicit... the added log currently prints regardless if the user passed the es module options or not, right?
| // When Wasm is enabled, force Platform.JS (Scala.js Wasm backend requires JS compilation) | ||
| val scalaOptions = | ||
| if (wasmEnabled) | ||
| ScalaOptions(platform = Some(Positioned.none(Platform.JS))) | ||
| else | ||
| ScalaOptions() |
There was a problem hiding this comment.
might be worth a log about this being implicit.
Also, this might clash if a user explicitly passes --wasm with --platform native, is this detected anywhere?
There was a problem hiding this comment.
added a log in Run.scala when the Wasm path is entered ("Wasm mode enabled: using ES module output on JS platform")
| } | ||
| } | ||
|
|
||
| if (TestUtil.fromPath("bun").isDefined) |
There was a problem hiding this comment.
are the bun tests on the CI?
There was a problem hiding this comment.
Deno and Bun tests are conditionally guarded by TestUtil.fromPath — they run only when the runtime is on PATH. GitHub Actions runners don't include Deno or Bun, so these tests are skipped in CI
There was a problem hiding this comment.
+1
there ought to be a way to set those up.
There was a problem hiding this comment.
note - this could be done as a follow-up, Deno and Bun support could be extracted to separate PRs (as suggested by @tanishiking) with tests done there.
I don't think we want those silently untested and subject to bitrot.
| } | ||
| } | ||
|
|
||
| if (TestUtil.fromPath("deno").isDefined) |
There was a problem hiding this comment.
Are the deno tests run on the CI?
There was a problem hiding this comment.
Deno and Bun tests are conditionally guarded by TestUtil.fromPath — they run only when the runtime is on PATH. GitHub Actions runners don't include Deno or Bun, so these tests are skipped in CI
| } | ||
| else if (emitWasm) { | ||
| // For WASM mode with ES modules, run node directly instead of NodeJSEnv. | ||
| // NodeJSEnv's stdin piping with "-" doesn't work with Input.ESModule. |
There was a problem hiding this comment.
this conversation seems unresolved?
|
|
||
| // Detects the major version of Node.js on PATH; cached for the JVM lifetime (lazy val). | ||
| // Returns None if node is not found or version cannot be parsed. | ||
| private lazy val nodeMajorVersion: Option[Int] = |
| nodeMajorVersion.foreach { v => | ||
| if (v < 22) value(Left(new NodeVersionTooOldForWasmError(v))) | ||
| } | ||
| val nodeFlags = if (emitWasm && nodeNeedsWasmFlag) List("--experimental-wasm-exnref") else Nil |
There was a problem hiding this comment.
We can add some options as implicit if user doesn't specify them and we know they are necessary to run the Wasm build. Just make sure the implicit stuff is logged, so that the user knows what's happening.
|
The |
|
Not necessarily in the scope of this PR, but - some docs for how to set up the Wasm runtimes and example snippets to run (maybe a dedicated guide) would be a great follow-up. |
Implements scala-cli issue VirtusLab#3316: integrate WebAssembly with Scala CLI. - `--wasm` CLI flag and `//> using wasm` directive to enable WASM output - `--wasm-runtime <runtime>` option and `//> using wasmRuntime` directive Supported values: node (default), deno - `--deno-version`, `--wasmtime-version`, `--wasmer-version` options and corresponding directives for pinning runtime versions - **Node.js** (default): runs Scala.js WASM output with `--experimental-wasm-exnref` flag, requires Node.js >= 22 - **Deno**: runs Scala.js WASM output
…imes - Move --wasm flag to dedicated Wasm help group with --help-wasm option - Simplify wasmOptions parsing with fold/toRight pattern - Add runtime validation with UnrecognizedWasmRuntimeError in directives - Auto-enable WASM when wasmRuntime directive is set - Update reference documentation Code style: simplify denoNeedsWasmFlag, explicit runtime match cases, clean type annotation, scalfmt
…smRuntime bun)
- Add BunNotFoundError with install hint
- Add integration test for Bun (conditional on bun being on PATH)
- Add actions/setup-node@v6 node-version:24 to all Linux integration test
jobs: the default Node.js on ubuntu-24.04 runners is too old for Scala.js
WASM GC (which requires Node.js >= 22). Matches docs-tests job which
already pins node-version: 24
Node 24 still ships V8 12.x where wasm-exnref is gated behind --experimental-wasm-exnref; the flag only flips to default in V8 13.x (Node 25+). The previous nodeMajorVersion < 24 guard therefore left Node 24 (the version pinned in CI) without the flag, which made any Scala.js WASM code using exception bytecodes, runtime throws, JS interop or Scala 3 @main fail at runtime. Same reasoning applies to Deno (Deno 2.x = V8 12.x). Until V8 13.x is the default everywhere, just always set the flag, there is no any overhead
…efactor into ScalaJsOptions - Replace "WASM" with "Wasm" per WebAssembly spec: contraction, not acronym - Fix nodeNeedsWasmFlag to be version-aware: only pass --experimental-wasm-exnref for Node < 25 (V8 12.x); Node 25+ has it enabled by default, Node 26+ may remove it - Remove Node/Bun pre-flight version checks; let runtime fail naturally on old versions - Remove else-if-emitWasm branch (not needed) - Refactor WasmOptions into ScalaJsOptions: jsEmitWasm and wasmRuntime are now fields of ScalaJsOptions at both build and CLI layers; CLI flags are now --js-emit-wasm and --js-wasm-runtime under the Wasm help group; WasmOptions classes removed - linkerConfig() now forces ESModule when jsEmitWasm=true - Update all integration test CLI flags to --js-emit-wasm / --js-wasm-runtime
- Log when --experimental-wasm-exnref is injected in runJs (Node.js < 25) - Log when Wasm mode enables ES module output in Run.scala - Improve comment in Wasm.scala directive to reference AmbiguousPlatformError for --wasm + --platform native conflict detection
004606c to
7899c1e
Compare
- runDeno: append --experimental-wasm-exnref to existing DENO_V8_FLAGS instead of silently replacing user-defined flags; log when flag is set - ScalaJsOptions.linkerConfig: warn when Wasm overrides user-specified --js-module-kind (forced to ESModule by Wasm backend)
Lost during Run.scala restructure for Wasm: restores the original case _: SbtFile => "" and case null => "" branches so SbtFile inputs are properly filtered from scala.sources JVM property.
| object WasmRuntime { | ||
| case object Node extends WasmRuntime("node") | ||
| case object Deno extends WasmRuntime("deno") | ||
| case object Bun extends WasmRuntime("bun") |
There was a problem hiding this comment.
All of them are JS runtimes (that embed Wasm engine).
maybe we wanna rename this to JSRuntime? and move into JS option group?
Implements scala-cli issue #3316: integrate WebAssembly with Scala CLI.
--wasmCLI flag and//> using wasmdirective to enable WASM output--wasm-runtime <runtime>option and//> using wasmRuntimedirective Supported values: node (default), deno, wasmtime, wasmedge, wasmer--deno-version,--wasmtime-version,--wasmer-versionoptions and corresponding directives for pinning runtime versionsNode.js (default): runs Scala.js WASM output with
--experimental-wasm-exnrefflag, requires Node.js >= 22Deno: runs Scala.js WASM output; if not found on PATH, downloads from GitHub releases via Coursier cache
Wasmtime / WasmEdge / Wasmer: return UnsupportedWasmRuntimeError pending upstream Scala.js standalone WASM support (Make Scala.js Wasm backend suitable for standalone Wasm VMs (a.k.a. support "server-side Wasm") scala-js/scala-js#4991)